It doesn’t happen as much anymore, but it wasn’t long ago that every time I showed slides explaining how Amazon tests every change they make to their website, some wisenheimer would pipe up, “Yeah, well Amazon has a lot of money, so they can afford to test.” I was forced to edify them by explaining that it’s actually the opposite: “Amazon tests, so they have a lot of money.”
Nowadays, everyone seems to have the testing religion. They test up and down, but are they testing the right things? When I talk to clients, the vast majority of their tests are about very small changes:
- Color, placement, or text on call to action buttons
- One image vs. another
- One headline vs. another
- One page layout vs. another
Now, understand, none of these tests are bad ideas. Yes, you should test these things. But my problem is that this is just about all I ever see tested. There is a whole word out there waiting for you–it just needs you to take some initiative:
- Remove a step from your navigation to conversion
- Add a step from to your navigation
- Replace text with a video
- Add text transcriptions to your videos
- Change the marketing message
You might read these and say to yourself, “Gee, those tests are hard!” And you would be right. It takes more than a couple of minutes to think through the ramifications of these tests. Instead, I see two things simultaneously going on, both of which are wrong:
- All of the testing concerns the easy stuff we discussed at the top of the post. We do testing, but it’s all limited to a single page with variations that take minutes to think up.
- We make all the changes I described in the second list–we just don’t test any of them. Think about it. The last time you overhauled your navigation or your marketing message or redid your content types, did you test any of it? Probably not. You just did the creative work and made the change live in production and assumed you had improved things. You’re not actually testing the biggest, riskiest changes that have the most potential for improvement and the most risk of failure.
It doesn’t have to be that way.
Instead of relegating A/B testing to the simple stuff, your biggest changes are the ones to give the most testing scrutiny. Make sure you test the big stuff and you’ll benefit a lot more than from testing only the small stuff.
